Amazon Kinesis

Amazon Kinesis is a scalable and durable real-time data streaming service to analyze data in real-time from multiple data sources. Its Amazon's fully managed service for collecting, streaming and analyzing data in the cloud.

There are four types of Kinesis Streams

• Kinesis Data Streams
• Kinesis Firehose
• Kinesis Video Streams
• Kinesis Data Analytics

Kinesis Data Streams:

We have got the EC2, mobile phones, Laptops, IOT which are producing the data. They are known as producers as they produce the data.

The data is moved to the Kinesis streams and stored in the shard. Each shard has a sequence of data records. By default, the data is stored in shards for 24 hours. You can increase the time to 365 days of retention using the IncreaseStreamRetentionPeriod opertaion. The way you pay is as per the running shard.

Once the data is stored in shards, then you have EC2 instances which are known as consumers. They take the data from shards and turned it into useful data.

then the useful data is moved to either of the AWS services, i.e., DynamoDB, S3, EMR, Redshift.

Kinesis Firehose

Kinesis Firehose is a managed service of AWS for delivering the real-time streaming data to destinations such as Amazon S3, Amazon Redshift, Amazon Elasticsearch

With Kinesis Firehose, you do not have to manage the resources. you configure your data producers to send data to Kinesis Data Firehose, and it automatically delivers the data to destination that you specified.

Like Kinesis Data streams, the data is produced by the producers and sent to Kinesis Firehose. you do not have to do the manual editing of the shards to keep the data like how you do in the Kinesis data streams. its completely automated.

Data can be analyzed by using the Lambda function. once the data is analyzed, it is sent directly over to S3 or any other location like Amazon Redshift.

To send the analyzed data to Amazon Redshift. First, you have to write to S3 and then copy it to Redshift.

Kinesis Video Streams

As the name suggests, Kinesis Video Streams is used to stream live videos from devices to AWS cloud, or build applications for real-time video processing or batch oriented video analytics.

You can use Kinesis Video Streams to capture massive amounts of live video data from millions of sources, including smartphones, security cameras, webcams, cameras embedded in cars, drones, and other sources.

The output audio or video data to ML services like SageMaker or Rekognition or any customized AV processing services, so we can analyze and process the AV streams.

Kinesis Data Analytics

Kinesis Data Analytics take stream like data, Kinesis data stream or Kinesis firehose as input and output. Data that pass through the Data Analytics is run through customized SQL and the results are send to output. This allows us the real-time analysis of data like real time reporting.

Amazon Kinesis Data Analytics reduces the complexity of building, managing, and integrating streaming applications with other AWS services.

Thank you for reading!!